Response to Stephen Holgate.

P rofessor Stephen Holgate comments on our statement that “the new UK Research Excellence Framework (REF) puts more emphasis on bibliometric data and less on peer review than did its predecessor” [1]. As the Chair of Main Panel A of REF 2014, he describes the informed peer review process in his domain. In our opinion, peer review can indeed be considered as an ideal way of evaluating research. This process can be informed by bibliometric indicators to variable extents [2]. In the case of REF, our conclusion is that bibliometric indicators have obtained a more pronounced position than in the former Research Assessment Exercise. For example, sub-panel 11 (Computer Science and Informatics) intended to make use of Google Scholar as a source of citation information in addition to Scopus (Elsevier) data provided by the REF team (http:// www.ref.ac.uk/subguide/citationdata/googlescholar/#d.en.78940). Since a suitable process for bulk access to Google citation data, however, could not be arranged, the panel decided not to use Google citations, but continued to rely on Scopus data. Such discussions between the panels and the REF team show, in our opinion, an increased awareness of the possible choices among bibliometric indicators. The choice for one type of data or another can have consequences for the assessment.